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as a Medical Device (SaMD) - Discussion Paper and Request for Feedback 

Dear Sir or Madam: 

The Combination Products Coalition (“CPC”) 1 welcomes the opportunity to provide 
comments on FDA’s “Proposed Regulatory Framework for Modifications to Artificial 
Intelligence/Machine Learning (“AI/ML”)-Based Software as a Medical Device (“SaMD”) - 
Discussion Paper and Request for Feedback” dated April 2, 2019 (the “Discussion Paper”). The 
CPC applauds FDA’s efforts to solicit feedback from stakeholders before formalizing the A I/ML 
regulatory framework in a draft guidance. Overall, the CPC believes that the Discussion Paper 
represents a positive step forward in clarifying FDA’s expectations for application of A I/ML for 
SaMD, which are not currently described in other guidance documents. 

The CPC has carefully evaluated FDA’s proposed approach on A I/ML and although we 
believe the proposed approach is well-constructed, we ask that the Agency consider the following 
general comments, as well as our specific responses to the questions posed by FDA, which are 
provided below. 

• The scope of applicability of A I/ML to healthcare is not yet known; however, based on 
industry’s early understanding, we can predict that this technology will have a fundamental 
impact on how we execute clinical trials, interpret clinical data, and help inform healthcare 
practitioners of ideal therapies for patients. The CPC notes that the scope of the Discussion 
Paper is quite narrow and is essentially limited to how the medical device industry should 
plan for post-market changes to A I/ML SaMD after clearance through a single premarket 
pathway (510(k) premarket notification). Class III AI/ML SaMD can and should be able to 


1 The CPC is a group of leading drug, biological product, and medical device manufacturers with substantial experience 
and interest in combination product issues. One of our top priorities is to work collaboratively with FDA on issues 
affecting combination products to advance our common mission: providing the best possible health care to patients. 
Our diverse, cross-industry membership permits the CPC to bring a special, broad, and unique perspective to these 
issues. 
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take advantage of this proposed regulatory framework. The CPC requests that FDA 
compose any future discussion papers on AI/ML to consider broader and more pressing 
issues around AI/ML. including use of AI/ML in interpreting real-world and well- 
controlled clinical data as well as human understanding and interpretation of AI/ML 
algorithms, and the extent to which those algorithms must be understood for the 
manufacturer and regulator to make an appraisal of safety and efficacy. 

• The CPC requests that FDA address the critical question of when a continuously adaptive 
AI/ML may require a premarket submission, as the proposed framework currently 
addresses only incremental learning algorithms with gating mechanisms for updates. Given 
the rapid advances in AI/ML in recent years, and its great potential applications to medicine, 
the CPC believes a framework for such software will be necessary in the near future. 

• The CPC further requests that any future discussion papers on AI/ML, as well as any other 
topic papers or communications related to digital health, involve the Center for Drugs 
Evaluation and Research (“CDER”) and the Center for Biologies Evaluation and Research 
(“CBER”) and include considerations around systems that achieve or influence use of drug 
and biological products. The CPC urges FDA to develop coordinated and consistent digital 
health policies across FDA centers to reduce regulatory burdens and support digital health 
innovation that ultimately helps patients. 

• As AI/ML has been developed and deployed by other industries where safety critical 
aspects must be considered (e.g., the automotive industry), the CPC strongly recommends 
that FDA leam from these industries to adopt best practices and apply the appropriate 
aspects of these practices to AEML systems for medical purposes. 

• The CPC recommends that FDA address the least-burdensome approaches for AI/ML 
SaMD in the future guidance, similar to language used throughout the FDA Software 
Precertification Working Model. Specifically, FDA should provide clarity in what least 
burdensome AI/ML submissions from industry look like throughout the product lifecycle. 
Clarity is needed to ensure that the submissions include “the minimum amount of 
information necessary to adequately address a regulatory questions or issue.” This approach 
will ensure good use of FDA’s resources and timely access of life-saving AFMF SaMD to 
patients who need them. 
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Responses to FDA’s Specific Questions 

Questions / Feedback on the types of AI/ML-SaMD modifications 

1. Do these categories of AI/ML-SaMD modifications align with the modifications that 
would typically be encountered in software development that could require premarket 
submission? 

Generally yes, the high-level categories of AI/ML-SaMD modifications align with the 
modifications that would typically be encountered in software development. However, 
additional specificity is needed when discussing the intended use of software. Intended use is 
a broad regulatory concept. We would suggest adoption of a term such as ‘software 
functionality’ to describe changes to functions and features of software. Often, a SaMD 
manufacturer might add a new function, but might not translate that function to a high level 
claim or intended use of the device. Additional detail around how industry should or must 
consider new software functions as new ‘intended uses’ should be provided. 

2. What additional categories, if any, of AI/ML-SaMD modifications should be considered 
in this proposed approach? 

The categories of changes described in the Discussion Paper are central to the AI/ML aspects 
of software functionality; however, it is possible that future SaMD will have AFML and non- 
AI/ML functions as well. The Discussion Paper does not explicitly discuss how changes to 
more traditional medical device functions and claims should be evaluated when a portion of 
the system is not related to AI/ML. 

3. Would the proposed framework for addressing modifications and modification types 
assist the development AI/ML software? 

While the proposed framework is useful in understanding FDA’s current thinking for AI/ML 
changes, the Discussion Paper does not directly account for past categorizations of software 
changes as outlined in FDA’s guidance, “Deciding When to Submit a 510(k) for a Software 
Change to an Existing Device.” This document contains several change categories and 
flowcharts that do not appear to have been considered within the Discussion Paper. Therefore, 
it is not clear how industry should plan for regulatory evaluation of post-market changes using 
both of these resources. 

To ensure the utility of the upcoming guidance, we ask that FDA include within the document 
a flow diagram that clearly depicts examples of AI/ML-bascd software modifications which do 
not require a new submission and those that will require a new submission. 

Questions / Feedback on GMLP: 

4. What additional considerations exist for GMLP? 

APML-based SaMD can be associated with drugs, whether as a drug-device combination 
product or when AI/ML-bascd SaMD otherwise references a drug. The Discussion Paper’s 
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proposed framework does not discuss how FDA would regulate AFML-based SaMD 
associated with drugs. 

It is presently unclear whether CDRH believes the considerations in the Discussion Paper apply 
in full to device-led, drug-device combination products with an AFML-based software 
component, where the software associated with a drug is not a device or where the software 
references drugs marketed by third-party drug manufacturers. The Discussion Paper also leaves 
open the question of whether FDA will regulate clinical and patient decision support tools that 
utilize AFML-based software that evolve over time and include recommendations about 
pharmaceuticals. 

In summary, the CPC recommends that FDA clarify how the proposed Discussion Paper 
interacts with CDER and CBER’s proposed approach to digital health issues, including 
Prescription Drug-Use-Related Software (“PDURS”). The CPC also recommends coordination 
between center-specific frameworks for regulating AFML-based software to ensure consistent 
treatment of AFML-based software products, including software disseminated by or on behalf 
of a drug sponsor. 

5. How can FDA support development of GMLP? 

Development of a GMLP should involve key stakeholders from industry, academia, and PDA. 
As such, PDA should consider holding public workshops and webinars to examine real-world 
case studies and encourage dialogue between the key stakeholders. This will ensure robustness 
and utility of the GMLP. 

Purthermore, we ask that PDA also consider the following CPC recommendations: 

• Endorse and further support proliferation of Office of the National Coordination for 
Health Information Technology (“ONC”) driven interoperability standardization 
efforts, and provide incentives for device manufacturers to participate and benefit from 
such activities (Past Healthcare Interoperability Resource (“FHIR”) proliferation etc.) 

• Support training for hospitals around good practices for data integrity, and perhaps start 
to regulate those practices so that all stakeholders can benefit from “clean” data. 

• Establish guidance for manufacturers on how to handle “user errors” and poor data 
integrity practices. 

• As AI/ML has been developed and deployed by other industries where safety critical 
aspects must be considered (e.g., the automotive industry), the CPC strongly 
recommends that FDA learn from these industries to adopt best practices and apply the 
appropriate aspects of these practices to AFML systems for medical purposes. 

• Provide clarification on how a manufacturer should apply existing design controls and 
lifecycle management principles (which are general and intended to apply to all 
devices) to AFML SaMD. 
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6. How do manufacturers and software developers incorporate GMLP in their 
organization? 

The FDA model for the Total Product Lifecycle (“TPLC”) Approach to the Regulation of 
APML-Based SaMD involves the following four elements: (1) culture of quality and 
organizational excellence through the proposed SaMD manufacturer pre-certification program, 
(2) premarket assurance and safety and effectiveness, (3) review of SaMD pre-specifications 
and change protocol, and (4) real-world performance monitoring. Relative to the current 510(k) 
regulatory paradigm, one should note that item (2) remains the same while items (1), (3), and 
(4) are new requirements placed on SaMD manufacturers. On the surface, the TPLC approach 
proposed by FDA for APML-based SaMD appears reasonable, though we are concerned that 
the actual implementation of this TPLC approach could be unduly burdensome for both FDA 
and SaMD manufacturers. 

For example, details of the pre-certification program to ensure a “Culture of Quality and 
Organizational Excellence” are at an early pilot stage based on a limited dataset of 
manufacturers. It appears that the purview of SaMD pre-specifications and change protocol 
will be a critical element during FDA review of AFML-based SaMD. However, SaMD 
sponsors and FDA have little experience submitting and reviewing this type of information. As 
FDA develops specific guidance related to AI/ML. the CPC strongly encourages FDA to post 
“real-world” examples and case studies to provide additional clarity for developers. 

Finally, 510(k) SaMD products currently do not have an annual reporting requirement and it 
appears that item (4) - Real World Data Monitoring - is a step towards 510(k) product up- 
regulation. The CPC recommends piloting of a Real World Data Collection Program to 
demonstrate its benefits relative to the increased regulatory burden to both FDA and the SaMD 
sponsor for mandatory post-market surveillance reporting for all SaMD products. In summary, 
FDA’s TPLC approach toward SaMD seems reasonable at first glance, but it appears that the 
approach is a significant “up-regulation” compared to the current 510(k) regulatory paradigm. 

Questions / Feedback on SPS and ACP: 

7. What are the appropriate elements for the SPS? 

FDA’s suggested principle of a “predetermined change control plan” is appropriate for AI/ML 
SaMD systems. In defining the types of anticipated modifications to the system, the SaMD Pre- 
Specifications (“SPS”), the CPC’s recommendations below align with FDA’s statements in 
Section II of the Discussion Paper, which note that “the rigor of performance evaluation for 
both locked and continuously adaptive algorithms depend on the test methods, quality and 
applicability of dataset used for testing, and the algorithm’s training methods,” and that 
“[r]obust algorithms typically require the availability of large, high-quality, and well-labeled 
training data sets.” 

Recommended elements for the SPS include: 

• Detailed description of the initial training sets used; 

• Detailed description of any additional training sets used after deployment; 
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• Documentation of the detailed data requirements; 

• Documentation of the defined “safe behaviors” within which the ML component must 
operate (i.e., what “boundaries” has the manufacturer placed on the system, based on 
their risk management plan, to ensure that the ML component(s) cannot operate in a 
way that would create a patient safety hazard); 

• Documentation of identified hazards and post-deployment hazards that have been 
identified; 

• Description of identified faults and failure modes and any that have been identified 
post deployment; 

• Description of the baselines used for comparison of any classifiers in any ML 
component(s); 

• Description of the methods and results of statistical significance testing of all ML 
components and the system; and 

• Description of anticipated change to the SaMD risk categorization, core functionality, 
or performance. 

8. What are the appropriate elements for the ACP to support the SPS? 

In addition to the Algorithm Change Protocol (“ACP”) overview provided in Figure 4 of the 
Discussion Paper, the CPC recommends that the Agency consider the following requirements 
for methods that a manufacturer should employ to appropriately control the risks of the changes 
made in the SPS: 

• Methodology to describe the error rate associated with an ML model, including 
statistical methods and results; 

• Method to establish how representative a training set is and risk management to address 
training sets that underrepresent safety-critical cases; 

• As training sets are not “specifications,” due to being incomplete by their nature, and 
training is not “verification,” traceability should flow from high level requirements to 
detailed data requirements that ensure that appropriate training, validation, and testing 
sets are obtained; 

• Use of fault detection tools and techniques that take into account the unique features 
of ML (e.g., complex interactions between linked ML components); 

o An example of a method would be to have redundant features use different ML 
models and training sets to detect potential failures when there is disagreement 
between the features; 

o Another example is to define a “safety envelope” of possible known safe behaviors 
and limit the ML component to choose only from among those; 

• Establish safety requirements for ML functionality that is inherently unspecifiable (i.e., 
with the knowledge that they have an error rate and will periodically fail); 

• Express requirements in terms of the intent and maturity of the technique being 
employed (i.e., because not all models are implemented using an imperative 
programming language); 

• Ensure that preprocessing is consistent and uniform in its approach to the removal of 
outliers, incomplete data, or noise; 
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• Ensure that representative data are free of sampling bias (techniques to mitigate bias 
should be used); and 

• Describe the methods and results of statistical significance testing. 

9. What potential formats do you suggest for appropriately describing a SPS and an ACP 
in the premarket review submission or application? 

The SPS and ACP should be submitted as part of the premarket review submission or 
application. FDA should place an emphasis on dedicated interactive review of such elements, 
enabling a SaMD developer to walk FDA through its proposals and iterating on changes to 
them prior to clearance or approval. The CPC recommends FDA apply a novel regulatory 
pathway for SPS and ACP to ensure a timely review process. For example, FDA’s “real-time- 
review” (“RTR”) regulatory mechanism, which routinely applies on PMA-approved devices, 
may be applied, with the exception that the RTR should be done in a much shorter timeframe. 

Questions / Feedback on premarket review 

10. How should FDA handle changes outside of the “agreed upon SPS and ACP”? 

In Section IV, Subsection 4 (transparency and real-world performance of AI/ML-bascd SaMD), 
the Discussion Paper states that “FDA would also expect the manufacturer to provide periodic 
reporting to FDA on updates that were implemented as part of the approved SPS and ACP, as 
well as performance metrics for those SaMD.” The CPC believes that if a SPS and ACP have 
been approved, the requirement for updating FDA seems redundant if the proposed approach 
in Figure 5 is to document this modification. The net effort of this requirement is notification 
instead of documentation. 

The CPC recommends that all SaMD manufacturers establish a robust quality system that is 
inclusive of good machine learning practices and software risk management principles. For 
changes outside of the “agreed upon SPS and ACP,” the CPC encourages FDA to apply a 
pragmatic, risk-based framework on lifecycle management for SaMD software. An effective 
regulatory framework that allows timely and efficient introduction of software changes is 
important to the quality, safety, and availability of innovative software products. It is 
acknowledged that software changes vary from low to high potential risk with respect to their 
impacts on product safety and/or efficacy. The regulatory reporting mechanisms associated 
with the changes should be commensurate with the potential risk. 

11. What additional mechanisms could achieve a “focused review” of an SPS and ACP? 

SaMD undergoes frequent refinements and, importantly, changes of Al/MF may occur in real¬ 
time. As such, the “traditional” regulatory pathway such as pre-submission is inherently not 
compatible with the rapid lifecycle management of SaMD. The CPC recommends that FDA 
utilize a novel regulatory pathway for SPS and ACP to ensure a timely review process. For 
example, as noted above, an RTR regulatory mechanism, which routinely applies on PMA- 
approved devices may be applied, with the exception that the RTR should be done in a much 
shorter timeframe. 
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12. What content should be included in a “focused review”? 

The CPC recommends focusing on substantive changes of the approved SPS and ACP only. 
As such, the “focused review” should focus on (a) description of the proposed changes, and (b) 
rationales and the outcome of the risk assessments. 

Questions / Feedback on the transparency and real-world performance monitoring: 

13. In what ways can a manufacturer demonstrate transparency about AI/ML-SaMD 
algorithm updates, performance improvements, or labeling changes, to name a few? 

The CPC recommends a risk-based approach to transparency requirements for AI/ML-SaMD 
modifications. Use of AI/ML-SaMD will generate vast amounts of data, and the adaptive, 
iterative nature of AI/ML-SaMD could result in continuous, incremental, minor modifications. 
The CPC recommends that transparency requirements consider when a significant “change” 
within FDA’s benefit-risk framework would trigger a notice or a premarket submission to 
ensure unnecessary, voluminous information is not provided to FDA. FDA suggests periodic 
reporting on updates implemented as part of an approved SPS and ACP as well as performance 
metrics for APML-based SaMD. FDA also suggests that updates may need to be shared with 
collaborators of the manufacturer and the public on changes that were implemented in 
accordance with the approved SPS or ACP. It will be important for manufacturers to understand 
when a change implemented within the scope of an approved SPS or ACP is sufficiently 
significant to trigger a reporting requirement, and when these will trigger a notice requirement 
to the user. The CPC recommends that FDA define how AI/ML-bascd SaMD modifications 
align with FDA’s benefit-risk framework and International Medical Device Regulators Forum 
(“IMDRF”) risk categorizations, which should inform the appropriate reporting requirements. 
For example, whether a modification is within the parameters of the approved SPS or ACP 
should also factor into the benefit-risk analysis. 

The CPC agrees that transparency is important in ensuring the public is informed on how the 
SaMD functions, when substantive changes or improvements occur, and any limitations to the 
software in order to ensure patient safety and trust in using AFML-based SaMD. For those 
changes that do not trigger a label change or safety analysis, FDA and the manufacturer should 
agree on which modifications require notification to users, the method of communication, and 
frequency of the notice. The complexity of the software algorithm, the risk categorization of 
the SaMD, and target patient population should be considered in determining what information 
should be provided to users, clinicians and the public, and how that information should be 
delivered. 

In addition to the reporting requirements, FDA expects manufacturers to commit to real-world 
performance monitoring of the AI/ML-based SaMD to further transparency. The CPC 
recommends that FDA clarify its expectations on real-world performance monitoring of 
AI/ML-bascd SaMD versus the real-world performance monitoring under the Pre-Cert 
Program. It is unclear whether manufacturers of AI/ML-bascd SaMD will need to pre-certify. 
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14. What role can real-world evidence play in supporting transparency for AI/ML-SaMD? 

Real-world evidence can play a role in helping to validate and test AI/ML-SaMD and 
modifications to AI/ML-SaMD. As use of AI/ML-SaMD generates data, that data can in turn 
be used to test the algorithm’s function to ensure continued reliability and trust in the SaMD. 
Additionally, the ability to communicate real-world evidence generated from use of the AI/ML- 
SaMD can be another important way to communicate its function and reliability to users, 
clinicians, and the public. 

15. What additional mechanisms exist for real-world performance monitoring of AI/ML- 
SaMD? 

Post-market safety surveillance and reporting requirements are additional mechanisms to 
monitor the use and safety of AI/ML-SaMD. In addition, FDA states that manufacturers will 
be expected to commit to a TPLC approach to real-world performance monitoring for AI/ML- 
SaMD; however, it is unclear how this may differ from the real-world performance monitoring 
under the Pre-Cert Program. The Pre-Cert Program approach is still unclear with respect to how 
this information will be made available to FDA. The CPC recommends that FDA continue its 
engagement with stakeholders to clarify how real-world performance monitoring for AI/ML- 
SaMD may have similar or different data elements from the Pre-Cert Program. 

16. What additional mechanisms might be needed for real-world performance monitoring of 
AI/ML-SaMD? 

The CPC recommends piloting a Real-World Data Collection Program to demonstrate its 
benefits relative to the increased regulatory burden borne by both FDA and the SaMD sponsor 
for mandatory post-market surveillance reporting for all SaMD products. 

Questions / Feedback on the ACP: 

17. Are there additional components for inclusion in the ACP that should be specified? 

• Consider using leave-one-out cross-validation as a mitigation strategy to latent 
information that is highly correlated to labels, annotations, or other supervised 
information; and 

• Limit channel effects (e.g., spurious correlations due to the manner data were collected: 
limited clinical settings, geographies, patient populations, etc.) using statistical 
techniques like factor analysis. 

18. What additional level of detail would you add for the described components of an ACP? 

N/A 
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We appreciate the opportunity to provide input on the proposed framework and are happy 
to meet with the Agency to clarify or discuss any of our suggested revisions. 

Yours truly, 

Bradley Merrill Thompson, 

On behalf of the Combination Products Coalition 



